The Comparison and Combination of Genetic and Gradient Descent Learning in Recurrent Neural Networks: An Application to Speech Phoneme Classification

We present a training approach for recurrent neural networks by combing evolutionary and gradient descent learning. We train the weights of the network using genetic algorithms. We then apply gradient descent learning on the knowledge acquired by genetic training to further refine the knowledge. We also use genetic neural learning and gradient descent learning for training on the same network topology for comparison. We apply these training methods to the application of speech phoneme classification. We use Mel frequency cepstral coefficients for feature extraction of phonemes read from the TIMIT speech database. Our results show that the combined genetic and gradient descent learning can train recurrent neural networks for phoneme classification; however, their generalization performance does not show significant difference when compared to the performance of genetic neural learning and gradient descent alone. Genetic neural learning has shown the best training performance in terms of training time.